Menu directory status & updates copyrights help

Wilhelm Abel, David Fischer - The Great Pricing Waves 1200-1990 AD

"... History never repeats itself, but it rhymes. ..." (?John Maynard Keynes?)

Table of Contents


NOTE : The model here is DEFINITELY NOT suitable for application to [trade, invest]ing! It's far too [early, incomplete, immature, erroneous]! See Steven Puetz's http://www.uct-news.com/ for the investment side of the UWS.


Model results

"SP500 & DJIA indexes back to 1872"

Linear semi-log segments [1872-1926, 1926-2020]

Fibonnaci levels versus log-levels
Larger Fibonnaci's at greater timescales

How do I get negative numbers from a log transformation of the data?



What's [wrong with, missing from] the current model?


Are [political, economic, philosophical, psychological, sociological] theories unimportant?

The easy answer is "YES", as these seem to pre-occupy everybody, and everybody seems to have their own ideas (which are almost invariably other people's ideas, which are almost invariably of origins in antiquity) to impose on others.

But a justifiable interpretation of the price revolutions, and the analysis of BOTH [Alfredo Pareto, Benoit Mandelbrot] that wealth distributes are the same across societies (slopes can differ), is that our "brilliant ideas count for far less than our egos tell us. This seems be a particular problem with intellectuals, perhaps because of their very [nature, priviledged positions].

While ideas seem to be important, it's easy to forget that they also can cause massive problems, and there is little indication that we can think this through reliably.

I almost suspect that with progress, machines may better derive models and concepts than humans, which might be better based on data rather than "conclusions-driven bias" of [fragile, incomplete] human intellect? In many cases, they could hardly do worse.


And for the future?

Picking up a point from the section below "Linear regression of price [revolution, equilibrium] segments" :
We don't know [how, when] the "modern price revolution" will end, so I've merely used the last data-point. It's probably going to be bad, but as Fischer states, the impacts on people of crashes from price revolutions have become FAR less traumatic over the last eight hundred years. I suspect that this is due almost entirely to technology and changes from local-to-global trade. These have generated increasing "better than the pharohs" wealth, far greater stability.


???



Was 2007-2009 the modern equivalent to the 1929 crash?


Was 2007-2009 the moderen equivalent to the 1929 crash, less the socialist abuse of the free market? ??MacKay?? reference ??MacKay??'s commments are certainly unconventional, but merit closer attention. Key points :

How can the Great Pricing Waves be correlated with 'dead' system?



Inspirations

Inspired by many historians and financial market analysts, most notably but not limited to (in order of appearance on the world stage) :
Ibn Khaldun "The Muqquadimmah"
?? Tijchevski? - 10 year cycles & economic activities
?? Kondriatieff - ~50 y (Mayan) super-cycles
?? Elliot's waves
Arnold J. Toynbee "A Study of History"
  • (the full 10-12 volumes, I was missing one or two, now I have only the abridged two-volume set

  • Ivanka Charvova - best concept I've seen of solar activity over 5-10 ky
    Steven Yaskell and Willie Soon "Grand phases on the Sun" - wonderful descriptions of the Maunder grand solar minimum and many scientists and others
    Harry Dent - the most successful 10-40 year [economic, financial] forecaster that I am aware of
    Robert Prechter's Socionomics, the first quantitative Sociology
    Peter Fischers "The Great Wave" - I forget which author (probably Stephen Puetz who used some of the data) mentioned Fischer's book, but it is a real gem, built on the meticulous work of numerous historians. I consider it to be one of the great historical works.
    Stephen Puetz, and especially co-authors [Glen Borchardt, Andreas Prokoph, ??],
  • and the system of 20+ Mayan calendars

  • Ray Dalio of ?BlackRock" Hedge fund "??" - nice historical analysis from the point of view of a financial industry leader



    References

    David Fischer does a superbv job of listing and discussing a huge number of references on historical pricingg and the issues that arise!!!
    I need to somehow put a lot of that here! (huge work, as Optical Character Rcognition (OCR) is often problematic for me, even though it was pretty good at work)


    Data, Methodolgy, Applications, Software

    "$d_Fischer" = "http://www.BillHowell.ca/economics, markets/Fischer, David 1996 The Great Wave/" for the internet, but of course it is whatever directory path that you have downloaded the files of this webPage to on own system, if you do download.

    Data sources


    Primary :
    David Fischer 1996 "The Great Wave, Price revolutions and the rhythm of history" New York, Oxford University Press, ISBN 0-19-505377-X

    I have NOT [purchased, read] :
    Wilhelm Abel 1935 "Agrarkrisen und Agrarkonjunktur: Eine Geschichte der Land und Ernährgswirtchaft Mitteleuropas seit dem hohen Mittelalter" Hamburg & Berlin 1966 appendix https://de.wikipedia.org/wiki/Wilhelm_Abel


    I have NOT yet used :
    https://historicalstatistics.org/
    many links to other data sources
    https://thehistoryofengland.co.uk/resource/medieval-prices-and-wages/

    Modern themes that illustrate this best :

    Methodolgy


    Photograph paper graphs :

    Collect digital images of graphs in the book

    GNU Image Manipulation Program (gimp) - Image preparation for each curve of each graph :

    GNU Image Manipulation Program (gimp) isn't the easiest, most intuitive software, but it's the kind of initiation-by-fire that later leads to you being happy that you didn't stick with lesser [image, video] software. I cannot provide a users' guide here, but I will make [random, scattered] comments on the process. Because gimp [wording, processes, menus] can be a bit "strange" it may help to have this as a guide (or not). Of course, you may be using an entirely different image processing sare, in which case my instructions are useless for YOU. But these instructions are actually for me, as there is no way I will remember which [software, tools, techniques, how-to] even six months later unless I use applications often (not usually the case).

    gimp - [rotate, crop] graphs from each cell-photo

    gimp - remove [adjacent, cross-over] curves for each curve-file :

    This step ensures that in the next section, the digitizelt graph curve digitizer, will capture most, if not all, of the curve automatically with one attempt, which can save you a lot of time and frustration.

    gimp : curve, make [continuous, mono-color]


    Digitize graph curves using https://www.digitizeit.xyz/ software


    IMPORTANT issues with digitization of curves

    Digitization of curves is not without its issues, and this is NOT limited to https://www.digitizeit.xyz/ software, although different software may be [more, less] robust with respct to these problems. Digitizelt probably has settings that can be adjusted for particular datasets, but I have not looked for that in my first use of the software (later, maybe much later...). In a nutshell, the digitization must be [robust, accurate] on the whole, and compromises are required in the software. "Peak shaving" (high and low) occurs, and high frequencies tend to be smoothed out, and these can be critical to some applications.

    I have NOT done this simple work! Yes- laziness creeps in, but it's just an initial stage of the project, and this can be done later (yeah right, but will I do it?)

    Warehousing graphical data

    "$d_webRawe""economics, markets/Fischer, David 1996 The Great Wave/1_Fischer 1200-2020.ods" :
    -add sheet for each graph-curve

    I typically use a LibreOffice Calc spreadsheets to [collect, rearrange, simple transform] data. For this project : 1_Fischer 1200-2020.ods

    For gnuplot, it is most convenient (albeit not necessary) use comma-delimited (csv-like) text data files. Just save the spreadsheet in that format, so now you have a data warehouse with the [convenience, power] of spreadsheets, and simple output usable by gnuplot.

    Set timePoints for transitions between price [revolutions, equilibria]

    The [first, key] step is to select the timePoints for transitions between price [revolutions, equilibria]. The endPoints of price revolutions was taken at the point wherrices stabilised following a price revolution "crash" and tended to drift lower slowly. Price equilibria endPoints were taken to be where significant crease first occurred, and the subsequent price revolution begins, somewhat slowly.


    These timePoints are somewhat "cloudy", and while I kept "close" to dates as usually specified in Fischer's book, I didn't take these as exact. Perhaps the actual price data are a better reference. I used the "consumables prices" as a basis for other curves (eg grains) for the medieval to modern periods, but it's br to treat each curvce separately.

    This is susceptible to serious bias in selecting the [start, end] dates for each segment. See the spreadsheet 1_Fischer 1200-2020.ods.

    For example, I initially selected the following end-points fREngland's consumable prices :
    year 1206.58 1373.55 1501.78 1661.06 1754.20 1803.78 1937.77 1999.74

    The year ~1926 was taken as the [start, end] point for my 1872-2020 detrend StockMkt Indices 1871-2022 PuetzUWS2011 [start, end] point, so I use it here as well. (23Feb2023 - original text said 1940, perhaps it is still like that?)

    We don't know [how, when] the "modern price revolution" will end, so I've merely used the last data-point.


    Do single-linear-regressions of price [revolution, equilibrium] segments

    This is easy with the spreadsheet - one column of regression results I use 10 year intervals per segment, but you only really need the [start, end] dates [-,+] 20 years. The extra 20 years extends the segments at both ends for visual clarity. For an example, see the spreadsheet 1_Fischer 1200-2020.ods, sheet "Fig 0.01 SLRsegments".

    Save the "SLRsegments" to a data file that canused by GNUplot. Example : Fig 0.01 line segments for GNUplots.dat. Notice that olumn titles can use free-format text, except for the comma, which separates columns.

    It would greatly reduce annoyance if this entire step was automated.

    I used my simple Qnial programs to do the linear regression :

    GNUplot - create graph of nominal data

    30Jan2021 - I'm too lazy to provide details at this time, but the good thing is that I used gnuplot scripting, so this part is perfectly well described by scripts (in this web-directory) and the datasets. Examples are : For this project I did not write bash scripts that create the gnuplot scripts, but I've done that in the past. It's a life-saver for repetitive work.

    [calculate, plot] detrended results for each time segment

    During the step "Do singular linear regressions for each time segment" described above, the QNial program generates Yest values for each digitized point. These values are copy-pasted to the "data warehouse" spreadsheet, where a simple formula calculates detrended values.

    Note that I calculate 10* the detrended log values, which was used a previous project for the S&P 500 stock market index. From that community I have adopted a [fractal, Fibonacci] scaling of the outputs on a [-5.7] interval. Fibonacci levels (and related Elliot waves) are very widely used in technical analysis for trading, and I apply it to the historical series as well.


    The main detrended plots are defined in GNUplot files :

    Simple template for working on a dataset

    08********08
    ddmmmyyyy Template for data analysis steps

    1. copy section to new date near start of this file
    2. copy-replace "Fig 5.02 ancient Greece [barley, olive oil] 450-150 BC"
    ......with the new photo-image-curve name
    3. careful to update information as you work
    NOTE : d_gimp="$d_PROJECTS""References/Fischer, David 1996 The Great Wave/gimped/" d_Fischer="$d_webRawe""economics, markets/Fischer, David 1996 The Great Wave/" However, most applications REQUIRE the full path!!

    +-----+
    gimp : crop graph, curve [separate out, make [continuous, mono-color]]
    /media/bill/Dell2/PROJECTS/References/Fischer, David 1996 The Great Wave/gimped/Fig 5.02 ancient Greece [barley, olive oil] 450-150 BC raw image.xcf

    /media/bill/Dell2/PROJECTS/References/Fischer, David 1996 The Great Wave/gimped/Fig 5.02 ancient Greece [barley, olive oil] 450-150 BC raw image.jpg

    +-----+
    digitizelt : digitize graph curves using https://www.digitizeit.xyz/ software
    $ java -jar "$d_SysMaint""digitizelt graph from image/DigitizeIt.jar"

    /media/bill/Dell2/PROJECTS/References/Fischer, David 1996 The Great Wave/gimped/Fig 5.02 ancient Greece [barley, olive oil] 450-150 BC raw data.dat

    Use an invert sort as number are negative! (use the spreadsheet if data crossed [BC, AD])
    $ sort --reverse "$d_gimp"Fig 5.02 ancient Greece [barley, olive oil] 450-150 BC raw data.dat" >"$d_Fischer""Fig 5.02 ancient Greece [barley, olive oil] 450-150 BC sorted.dat"

    Add column titles : must be QNial phrases - replace [space, punctuation]s with underscore `_
    +-----+
    QNial : process digitized data

    add_logCol link d_Fischer fNameBase ' sorted.dat'
    paste to : graph-curve sheet "Fig 0.02"
    column titles : must be QNial phrases - replace [space, punctuation]s with underscore `_
    column separator : tab

    copy log formula down "log_price column
    copy : titled-data ["i_dat "year "Fig_5_01_prices "log_price]
    to : "$d_Fischer""Fig 5.02 ancient Greece [barley, olive oil] 450-150 BC, QNial format.dat"


    +-----+
    Set timePoints for transitions between price [revolutions, equilibria], Linear regression of price [revolution, equilibrium] segments

    onset timePoints :
    equilibrium revolution equilibrium last_data
    -1796.061 -1739.991 -1684.230 -1624.393

    +-----+
    Do singular linear regressions for each time segment

    Use "$d_Fischer""Fig 5.01 linear regression raw data.dat"

    "$d_Fischer""Fig 5.01 prices in ancient Babylon 1850-1600 BC.dat"

    +-----+
    GNUplot - create graph of normal data

    "$d_Fischer""Fig 5.01 prices in ancient Babylon 1850-1600 BC.plt"

    +-----+
    [calculate, plot] detrended results for all curves analysed to date

    "$d_Fischer""1850 BC to 2020 AD prices detrended.dat"


    Software tools - general comments

    Here are the key [image, graph] tools that come to mind :
    https://www.digitizeit.xyz/ to extract data from digital images of graphs
    https://www.gimp.org GNU Image Manipulation Program - [free, very powerful, etc, etc]
    I have used gimp "scripts" - mostly for video aimation production
    http://www.gnuplot.info/ GNU plot - [free, very powerful, etc, etc]

    My basic toolset for much-of-what-I-do : My requirements for software :